Comment on "Finite Size Scaling in Neural Net- 
works" 
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Nadler and Fink use finite size scaling (FSS) in [1] to 
estimate the storage capacity of Ising neural networks 
from the behavior for small N of P(a,N), the proba- 
bility that aN patterns can be loaded onto a network 
of size N. For the perceptron they obtain a capacity of 
a c = 0.796 ± 0.010, implying that the analytical result 
a c w 0.833 (Ref. [26] of the letter) is wrong. This is re- 
markable since it is hard to see how the latter result might 
be corrected within the replica theoretic framework used 
in its derivation. Our comment, however, demonstrates 
that FSS cannot be used to extrapolate to large N from 
the small system sizes considered by Nadler and Fink. 
Consequently their work does not provide evidence that 
the analytical result is incorrect. 

By definition P(a,N) is monotonic in a. If a and N 
are such that the equivalence between the two quantities 
established by FSS holds, P(a, N) must be well approx- 
imated by a function monotic in N for fixed a. The 
nonmonotonic behavior of P(a, N) is shown in Fig. 1 
for ai = 0.8. Thus it is methodologically flawed, to ap- 
ply FSS in the range of small N (N < 30) considered 
in the letter. Even for a = 0.810, we have found that 
P(a,N) increases with the system size (Fig. 1). Rea- 
sonably this can be taken as evidence that a c > 0.810, 
a finding which is hardly compatible with the estimate 
of Nadler and Fink. It is however difficult to see, how 
a precise extrapolation to the infinite system might be 
obtained from such small values of N. 

We have been able to consider somewhat larger sys- 
tems than in the letter since we do not use exhaustive 
search. Instead we exploit the fact that the loading prob- 
lem for the perceptron may be efficiently decided by the 
simplex algorithm in the case of continuous couplings: 
Consider a binary search tree of depth N + 1 and iden- 
tify a node of depth M + 1 with an assignment of 1 or 
— 1 to the first M couplings (J%, . . . , Jm). At such a node 
one may ask whether continuous couplings Jm+i, ■ ■ ■ , Jn 
in the interval [—1,1] exist such the the entire perceptron 
(Ji, . . . , Jjv) stores a given training set. If not, the sub- 
tree of the node may be discarded. Traversing the search 
tree but pruning subtrees in the described manner yields 
an algorithm which decides the loading problem for bi- 
nary couplings. For values of a around 0.8 we have found 
the time complexity of this algorithm to scale roughly as 
2 N ' 3 , much better than exhaustive search. This has en- 
abled us to estimate P(a, N) rather accurately by con- 
ducting at least 3 x 10 4 trials for each data point. As in 
the letter, training sets with Gaussian inputs where used. 


[1] W. Nadler and W. Fink, Phys.Rev.Lett. 78, 555 (1997). 
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FIG. 1. P(a,N) as a function of N for a 
(upper graph) and a = 17/21 « 0.810 (lower 
values for the lower curve are: P(17/21, 21) = 0. 
and P(17/21,42) = 0.6696 ± 0.0020. Standard 
shown. 
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= 4/5 = 0.8 
graph) . The 
6660 ±0.0007 
errorbars are 
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